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AUTOMATED COLOR MATCHING FOR TILED PROJECTION SYSTEM 

TECHNICAL FIELD OF T HE INVENTION 

This invention relates to projection display 
systems, and more particularly to display systems that 
use multiple projectors to generate a tiled display and 
to a subsystem and method for matching the colors 
generated by each projector. 
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BACKGROUND OF THE INVENTION 

A tiled display is used when a large display is 
desired. The outputs of several projectors are tiled so 
that each projector provides a portion of the display. 
5 For example, the output of six projectors could be tiled 
to provide a two -by- three array of images, each image a 
portion of the total image. In the case of tiled 
displays, the color space of each projector must be 
matched, using color correction techniques. 
^ 10 Color correction is designed to correct for 

& variations in the color spaces of two different imaging 

devices. Color correction changes the color values of 
y| the pixels in an image in a way such that what the eye 

sees is consistent. 
V\ 15 A first type of color correction is the correction 

i*: of the display system to the device used to capture the 

w image. The optical system of each projector has its color 

ILLi 

filters adjusted and tuned to provide a match to other 
projectors. 

20 A second type of color correction is electronic 

color correction. Electronic color correction adjusts 
the input signal to the display device to match the 
projectors. Each display to be matched must be measured 
to determine the color range or gamut over which it 

25 operates. From this data, (for each display), all 
displays can be corrected to match. This method is 
easier to implement than optical system adjustments 
because it requires only measurement of the systems for 
color correction to occur. 
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SUMMARY OF THE INVENTION 

One aspect of the invention is a method of color 
matching images generated by multiple projectors of a 
tiled projection display system. It is assumed that each 
5 projector has a processing unit and that the system has 
some sort of main controller. The main controller could 
be one of the projector processing systems acting as a 
master controller, or it could be a separate processing 
system in communication with the projector processing 

10 systems via a bus. Each projector stores its own 

chromaticity data, this data representing at least the 
color gamut of images generated by that projector. It 
may also represent various aspects of luminance of the 
images. Each projector communicates this chromaticity 

15 data to the main controller. The main controller stores 
or computes a standard color gamut data, which represents 
a standard color gamut to which the projectors are to be 
matched. The main controller uses this standard color 
gamut data as well as the chromaticity data from each 

20 projector, to calculate color correction data for each • 
projector. The main controller then communicates each 
projector's color correction data to that projector, 
which uses the color correction data to calculate pixel 
values . 

2 5 The data stored at each projector and communicated 

to the main controller may also include data representing 
relative luminance of colors, such as might result from a 
color wheel or other color filter. The data may also 
represent luminance characteristics of a light source. 
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An advantage of the invention is that it avoids the 
need for tedious manual measurements and adjustments 
after the system is installed. The color and luminance 
data for each projector are stored with that projector 
and the color matching is automatically performed. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

FIGURE 1 illustrates the projectors of a tiled 
projection system and the tiled display. 

FIGURE 2 illustrates a typical data flow of the 
image data within each projector. 

FIGURE 3A illustrates the internal components of a 
projector having a color wheel and a single SLM. 

FIGURE 3B illustrates the internal components of a 
projector having multiple SLMs . 

FIGURE 4 illustrates how the color correction matrix 
for each projector is calculated and then used for the 
color matching process of FIGURE 2. 

FIGURE 5 illustrates how a standard color gamut is 
derived from the color gamuts of the projectors. 
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DETAILED DESCRIPTION OF THE INVENTION 

FIGURE 1 illustrates the projectors 11 and main 
controller 13 of a tiled projection system. These are 
the elements of the projection system relevant to the 
invention; a typical display system has various other 
electrical and mechanical features. 

. The images from four projectors 11 are projected 
onto a screen 12. Each of the projectors 11 contributes 
a tile of the screen 12. Thus a first projector 
generates tile 1, a second projector generates tile 2, 
etc. In- the example of FIGURE 1, the total image has 
four tiles. 

As explained below, each projector 11 is a SLM 
(spatial light modulator) projector. In the example of 
this description, each projector 11 has a single SLM and 
uses a color wheel to provide color images. Further 
details of projectors 11 are set out below in connection 
with FIGURE 3. In other embodiments, color could be 
provided by using two SLMs and a color wheel with a color 
splitting prism, or with three SLMs. Each SLM generates 
images of a different color and the images are optically 
combined to create a full color images. 

Each projector 11 stores certain color and luminance 
data associated with that projector 11. This data 
includes at least three types of data. Projector 
chromaticity data, M, represents various aspects of color 
and luminance of the projector. As explained below, 
although this data may represent certain luminance 
aspects of the projector as well as its color, it is 
referred to herein as "chromaticity" data to distinguish 
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it from other types of luminance data. Relative 
luminance data, E, represents the relative luminance of 
each color of the color wheel relative to a base rate at 
which the color wheel is running. Overall luminance 
data, L, represents characteristics of its illumination 
source. As explained below, this data is provided to 
main controller 13, which returns color correction data 
to each projector 11. For each projector 11, its color 
correction data, CM, represents the correction needed for 
that projector 11 to match its color to a standard color 
gamut. Each projector 11 then uses its color correction 
data to match its pixel data to that of the standard. 

Main controller 13 performs various control 
functions required for generating the display, including 
calculation of the color correction data for each 
projector 11. A bus 14 carries data. between projectors 
11 and main controller 13. 

FIGURE 2 illustrates a typical data flow of the 
image data within a projector 11. In the example of this 
description, it is assumed that the input data is YCrCb 
data, such as that provided by a television signal. Each 
projector 11 receives the data appropriate for its tile 
of the total image. Although not shown in FIGURE 1, it 
is assumed that the projection display system has some 
sort of receiver or data storage device for providing the 
input data . 

A first color space conversion (CSC1) process 21 
converts the YCrCb color space to an RGB (red, green, 
blue) color space. A degamma process 22 removes the 
gamma from the signal, such that the RGB signal is in 
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linear space. In other embodiments, the input data might 
already be RGB data, with no need for process 21. 

An RGBY gain process 23 applies gain to the signal. 
Process 23 is essentially for white processing, for 
projectors 11 whose color wheels have one or more white 
segments. 

A second color space conversion process (CSC2) 
process 24 converts the gained RGB signal to a color- 
corrected space in accordance with the invention. In 
other words, process 24 results in pixel data that is 
color matched between projectors 11. Further details of 
process 24 are described below. Process 24 may also 
include luminance matching, which compensates for 
variances in the illumination systems of the projectors. 

Process 25 applies error diffusion and RGBY offset. 
Error diffusion provides the correct number of bits per 
pixel for the display. The RGBY offset is an additional 
part of the white processing initiated in process 23. 
The resulting RGBW data is the input data for the 
projector's SLM. 

FIGURE 3A illustrates the internal components of 
each projector 11, where projector 11 uses a color . wheel. 
In the example of this description, each projector 11 is 
a digital micromirror device (DMD™) projector. Each 
projector 11 has a single DMD 31, which is a type of SLM. 
The DMD 31 has an array of tiny mirrors, each of which 
are individually addressable with image data. Each mirror 
can be tilted to either an on or an off position. An 
image is generated by illuminating the DMD 31 with light 
from a white light source 32, switching the appropriate 
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mirrors on or off and by modulating the time that the on 
mirrors are on. Images can be generated in such fast 
sequence, that the viewer perceives mot ion . 

Each projector 11 has a color wheel 33 for adding 
color to the imagoes. The color wheel 33 has red, green, 
blue, and white (RGBW) filter segments and spins in 
sequence with the images' corresponding RGBW data. Thus, 
for purposes of this example, a color wheel 33 having one 
or more white segments is assumed, but the same concepts 
could be applied to projectors without this feature. 

Mirrors are turned on or off depending on how much 
of each color is needed per pixel. The projector 11 
further comprises the color wheel motor 33a, pixel 
processing unit 34, various optics 35 and 36 for shaping 
the illumination to the DMD, and a projection lens 37. 
Processing unit 34 has one or more processors and 
associated memory appropriate for performing the 
processes of FIGURE 2. 

FIGURE 3B illustrates the internal components of 
projector 11, where projector 11 uses three SLMs (here 
DMDs 31) to generate color images. Each DMD 31 generates 
images of a different color. Optics 39, including a TIR 
(total internal reflection prism) , are used to combine 
the images into projection lens 37. 

Referring to FIGURES 1 - 3B, each projector 11 
delivers its stored color and luminance data (its 
chromaticity data, M, its relative luminance data, E, and 
its luminance data, L) to main controller 13. For each 
projector 11, main controller 32 calculates color space 
correction data, CM, which it delivers to the processing 
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unit 34 of that projector. The projector 11 uses this 
data during the color and luminance matching process 24 . 

Given a set of projectors 11 to be matched, the 
projectors 11 could be polled to get the color and 
luminance data of each. Additionally, controller 13 
could communicate with customer interfaces and provide 
inputs for adjustment of the white point of the system. 
Or, it could provide outputs of the lamp status of each 
projector 11, using the luminance data collected from 
each projector. This could give the customer advanced 
warning to the need to replace lamp 32 prior to lamp 
failure. 

Main controller 13 could be a DSP, a 
microcontroller, or an external computer such as a PC, or 
workstation. It could be placed within a bus structure 
14, which could communicate* with each projector 11, 
polling for the needed color and luminance data and then 
returning the color correction data. Alternatively, one 
of the projectors 11 could have a master processing 
system that performs the role of main controller 13 . 

Calculation of Color Correction Data 

FIGURE 4 illustrates how the color correction data, 
CM, is calculated and used. In the example of this 
description, the chromaticity data and the relative 
luminance data are calculated in matrix form, designated 
as the matrices M and E, respectively. The color 
correction data is returned to the projectors in matrix 
form, designated as CMj, for each projector 11. 



AUS01 =217959 .1 



ATTORNEYS DOCKET PATENT APPLICATION 

TI-28576 
(032350. B187) 

11 

As illustrated, Steps 41 - 43 are performed for each 
projector 11, resulting in the stored M and E data for 
each projector 11. Steps 44 - 47 are performed by main 
controller 13, using the M and E data to calculate CMj for 
each projector 11. Step 48 is the actual correction of 
the RGBW data at each projector 11, using the CMj data 
from the main controller 13. 

Step 41 involves measuring chromaticity and relative 
luminances for each projector 11. For each projector 11, 
the following chromaticity measurements are obtained, 
using conventional measurement equipment for measuring 
chromaticity (X, Y, Z) values off the screen: 



^G ' ' ^G 

y v 7 

^ B » 1 B'** B 

, where the subscript WS indicates white. Luminance 
measurements are also acquired as follows: 



1 RGB 



Y' 

1 w 



The following chromaticity terms are defined: 



X rgb - % R + X G + X B 
Yrgb ~ Yr + Y c + Y B 
Zrgb ~ Z R + Z G + Z B 



In the example of this description, the relative 
luminance data, E, is based on effective light times 
(ELTs) of color wheel 33 at a defined standard rate, such 
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as 60 cycles per second. In some systems, discrepancies 
between the ELTs at various rates may be insignificant 
and the measurements and calculations involving the E 
data could be omitted. In other embodiments, such as 
those using multiple SLMs to generate color images, the 
relative luminances could be measured for the images 
generated by the different SLMs. 

The ELT ! s from a standard rate of color wheel 33 are 
expressed as : 



ELT* , ELT* , ELT* , ELT } 



ws 



The chromaticity measurements can be adjusted for 
additional wheel rates: 

n _ ELT t 
*~ ' * ELT* 

Y\=Y, V 

* *■ ELT* 

N ELT * 

9 ' ELT* 

. y where <p = R,G,B,WS ; ELT+ is the standard wheel rate ELT 
for a given color <(); ELT^ is the ELT for wheel rate N for 

a given color <(>; and X%, Y N ^ 7?$ are the new adjusted 
measurement for the given wheel rate N. 

The above measurements may also be used for various 
calibration adjustments for the projector 11. 

In Step 42, the above measurements are used to 
calculate a chromaticity matrix, M, and a relative 
luminance matrix, E, for each projector 11. First, the 
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chromaticity and luminance data is used to calculate two 
parameters, a and P, and to compute white segment 
color imetry. 



Y' * Y 

a _ 1 RGB 1 WS 

Y f • y 

1 WS 1 RGB 

^ws = a * ^ws 



Y' 
Y' 

1 RGB 



The chromaticity, luminance, and white segment data 
is then used to form matrix A as follows: 



SfiGB ~ ^ RGB + Y RGB + ZflGB 



$g ~ X c +Y G + Z c 
Sws ~ + Y ws: + Z 



L WS 



WS 



"WS 



$b ~~ X B +Y B +Z B 



A = 













s B 












s B 















The inverse of A is calculated as: 



Define : A = 
Then 



a b c 
d e f 
g h i 



A' x =Dt- 



e-i-f-h 
f-g-d-i 
dh-e- g 



Dt = 



c-h-b-i b- f -ce 

a-i-c-g t cd -a- f 

bg-ah a-e-bd 
1 



(a-e-i-a-f • h-d bi + d -c- h + g -b- f -g-ce) 
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Normalized white points are calculated as: 



RGB, 



1 RGB 



Wy=l 

J$r _ ZrgB/ 



1 RGB 



WS x =^* S /yn 
/ X WS 

WSy=\ 

UZQ _ ^WS / 

/ X WS 



Define : A' 1 = 



a 


b 


c 


d 


e 


f 


g 


h 


i 



K R =W x a + b + W 2 c 
K G =W x -d + e + W z -f 
K B =W x g + h + W 2 -i 

The chromaticity matrix, M, for each projector 11, is 
calculated as: 



M = 



a b c j 
d e f k 
g h i I 



, where the letters a - 1 are placeholders and not the 
same as in the A matrix above. Values for each of these 
placeholders are calculated as follows: 



F = K */ 

Fg = Kg As g -elt g ) 

** /(S B -ELT B ) 



Y' 

y 

RGB 



a = F R -X R b = F c -X c 



c = F B -X B 



'WS 



WS y 



WS 
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d = F r Y R e = F G -X c f = F s .Y B k = 

Z' 

g = F R 'ZR h = F c -X c i = F B' Z B l = E^-~- 

A relative luminance, matrix, E, can be derived for 
each projector 11 from its ELT data as follows: 

'eltH 0 0 

0 ELTq 0 
0 0 ELT* 

, where N is a given color wheel rate. 

In Step 43, the color and luminance data for each 
projector 11 is stored in memory of its processing unit 
34. This data includes the above -described matrices M 
and E, which are to be subsequently downloaded to main 
controller 13. Alternatively, data used to derive these 
matrices may be stored, then downloaded to main 
controller 13, which calculates the matrices. In some 
configurations of projectors 11, the color wheel 33 has 
an associated EPROM, where this data may be stored. Each 
projector 11 now has its own colorimetry stored in its 
memory . 

Step 44 is to compute a standard color gamut, which 
will be used as the standard to which all projectors 11 
are matched. This can be accomplished by finding the 
largest color gamut triangle that is contained within the 
color gamut of all the projectors 11. This color gamut 
can be found by pairing two projectors, and finding the 
largest color gamut which is contained within their color 
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gamut. This new color gamut is then paired, with all 
other projectors, and a new gamut established from each 
pairing . 

FIGURE 5 illustrates the process of determining the 
standard color gamut. As shown, the standard gamut is 
the largest color gamut completely contained within four 
projector gamuts. 

Referring again to FIGURE 4, in Step 45, the 
standard color gamut is used to determine a standard 
color matrix, M T . Known techniques may be used to 
calculate the matrix from the gamut. These techniques 
generally involve selecting three points from the 
standard gamut and a white point. The white point can be 
defined as either the average white point of all 
projectors 11 or as an arbitrary white point. 

Step 46 is calculating a color correction matrix, 
CMj, for each projector 11. Given the chromaticity 
matrix, Mj , for each projector: 



a b c j 



Mj= d e f k 
g h i I 



, the RGB portion of the matrix is: 



a b c 



Define : C = 



d e f 
g h i 



e-i-f-h 

fg-d-i 

d*h-eg 



c-h-b-i 



b- f -ce 
c-d-a- f 
ae-bd 



C" 1 = 



Dt- 



b-g-a>h 



AUS01: 217959.1 



ATTORNEY'S DOCKET 
TI-28576 
(032350. B187) 



17 



PATENT APPLICATION 



1 



(a-ei — a- f -h-d-b-i + d-c-h + g-b- f-g-c-e) 



Given the relative luminance matrix, E, for each 
projector 11, then: 



The color correction data, CM, for each projector j, 
may be expressed in matrix form as: 

CM . =E - l -C~ l 'C„'E T 
J J j T T 

, where E T is a standard relative luminance matrix derived 
from the standard wheel rates described above in 
connection with Step 41. 

Referring again to FIGURE 2, the next issue is the 
luminance levels of- the projectors 11. As part of 
process 24, following color correction, the luminance 
level of each projector 11 is matched. This can be 
accomplished using either a direct measurement of each 
projector 11, or through use of a sensor internal, to each 
projector 11. This internal sensor could be a sensor 
placed in a dump light path or other suitable location in 
which the light is proportional to the lamp output. This 
sensor could be calibrated in the factory to read a 
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number related to the luminance level of the projector 
11. The luminance data, L, of each projector 11 can be 
used to adjust the gain levels of the CM matrix. 

Additionally, information about the screen color 
performances could be input into main controller 13. 
Color correction for screen colorimetric performance 
could be performed. This could include correction for 
reduced blue saturation, or general white point movement 
due to the screen. 

Referring again to FIGURE 3, the color correction 
matrix, CM, for each projector 11 can now be downloaded 
into that projector 11. Each projector 11 uses this 
matrix to perform the color matching process 24 of FIGURE 
2. As a result, its images are color corrected into the 

gamut defined by C T -E^. 

As stated above, the relative luminances of 
some systems may not be a significant factor in color 
discrepancies between projectors 11. In this case, the 
color correction matrix could be more simply calculated 
as : 

CMj = Cj" 1 • C T 

In still other embodiments, the relative luminance 
data could be incorporated into the C matrix. 

The above is but one example of a color correction 
matrix. U.S. Patent Serial No. 09/561,160, entitled 
"Enhanced Color Correction", and incorporated by 
reference herein, describes an alternative color 
correction matrix. As in the present invention, the 
color correction matrix is a 3 x 3 matrix, but is derived 

AUS01 :217959 .1 



ATTORNEY'S DOCKET 
TI-28576 
(032350. B187) 



19 



PATENT APPLICATION 



from a 3x7 matrix that includes secondary color values as 
well as primary color values. Using the enhanced color 
correction technique provides additional flexibility in 
the color correction of multiple display systems. .This 
includes larger color gamuts and brighter displays. 

The above-described techniques are not limited to 
display systems using the digital micromirror device 
(DMD) . They are also applicable to other display 
technologies, such as LCD, plasma, CRTs, FEDs, laser 
illumination systems, or LED illumination systems. The 
concepts are applicable to both rear and front projection 
display technologies. 

Other Em bodiments 

Although the present invention has been described in 
detail, it should be understood that various changes, 
substitutions, and alterations can be made hereto without 
departing from the spirit and scope of the invention as 
defined by the appended claims. 
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